Proactive Storage Health Management to Reduce Data Center Downtime

The project adopts machine learning algorithms to perform proactive storage health management to reduce data center down time. Specifically, we develop reliable algorithms that can predict disk drive failures for timely replacement. The dataset is a collection of SMART disk drive attributes provided by BackBlaze. To construct compact time-series for model development, three techniques are used to process SMART attribute data, including time-series averaging, feature selection, and class balancing. Models based on four algorithms, including logistic regression (LR), support vector machine (SVM), extreme gradient boosting (XGBoost), and recurrent neural network (RNN), are developed and evaluated using prediction accuracy and F-score along with precision and recall. The baseline model LR achieved 87.2% prediction accuracy and a F-score of 0.87 with relative low recall of 0.82 for failed disks. Both prediction accuracy and F-score are improved in SVM, XGBoost, and RNN. Specifically, XGBoost and RNN achieved 94.9% and 94.1%, respectively with F-scores above 94%. More importantly, recalls of failed disks are improved to above 93% to greatly reduce the risk of false negative prediction. Finally, proactive prediction capability is explored to provide replacement suggestion in advance. RNN is able to keep prediction accuracy above 70% with 9 days of proactive time.